Method for providing convex piecewise-linear expression for multiple variable system

ABSTRACT

A method for providing a convex piecewise-linear expression which can be used for a system having thousands of variables is described. Data regions are defined and planes fitted to the data with the limitation that no plane exceeds the data by more than a set tolerance. A set of planes is formed and planes are added to the set one at a time found on the amount error reduction. After a plane is added to the set refitting occurs and another plane is selected for inclusion into the set.

FIELD OF THE INVENTION

The invention relates the field of fitting mathematical expressions todata.

PRIOR ART

There are many known techniques for fitting both continuous functionsand piecewise-linear expressions to data. These techniques have becomemore important recently since with current computing power it is nowpossible to rapidly solve mathematical expressions that were notsolvable a few decades ago. Moreover, as the cost of computing powerdecreases new opportunities arise for solving problems in unexploredfields.

Where a few variables and tens or hundreds of data points are involved,techniques using heuristic approaches and intuition may be adequate. Itis known, for instance, to use a piecewise-linear fitting representingthe characteristics of transistors where the maximum value of all of thepiecewise planes approximate the data. The breakpoint of the planes areselected heuristically since the operation of transistors are relativelysimple and well understood. On the other hand, a complex problem such asairline scheduling may involve hundreds of thousands of variables withno intuitive guidelines to set initial bounds. There are countless othersystems having thousands of variables which defy an intuitive approach.Thus there is a continuing need for rigorous methods that allow for thedeveloping of expressions especially for complex systems.

Additionally, mathematical expressions that can be expressed as, orconverted to, posynomial expressions lend themselves to geometricprograms. These programs can be solved with efficient interior-pointmethods. This technique is known particularly for solving electricalcircuit problems. See “CMOS Operational Amplifier Design In OptimizationVia Geometric Programming,” by Hershenson, Boyd and Lee, Proceedings ofthe First International Workshop on Design of Mixed Mode IntegratedCircuits and Applications, July 97 and “Automated Design ofFolded-Cascode Op Amps with Sensitivity Analysis,” by Herschenson, Boydand Lee, 5^(th) IEEE International Conference on Electronics, Circuitsand System, September 98.

SUMMARY OF THE INVENTION

A computer implemented method for providing a mathematicalrepresentation of a system having a plurality of variables is disclosed.Planes are fitted to the data where each plane deviates from the data byno more than a predetermined tolerance in one sense. A plane is selectedfor inclusion in a set of planes if inclusion reduces the error betweenthe set of planes and the data. After a plane is added to the set,refitting occurs and another plane is selected for inclusion in the set.This continues until there is no reduction in error from adding planesto the set. The maximum value of the planes in the set forms anexpression which represents the data.

In an alternate embodiment a look ahead method is used which determineswhether the inclusion of a particular plane in the set provides for moreerror reduction when taking into account the inclusion of planes insubsequent iterations.

In addition methods are described for providing a max-monomialapproximation as well as a posynomial approximation.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a flow diagram illustrating the major steps in the method ofthe present invention.

FIG. 2 shows in more detail one of the steps (step 14) of FIG. 1.

FIG. 3 is a flow diagram illustrating the steps associated with thefitting of planes to regions of data.

FIG. 4 is a flow diagram illustrating the steps associated with the lookahead method where n=2.

FIG. 5 shows a curve representing data and a single plane spaced-apartfrom the data.

FIG. 6 illustrates a single plane fitted to the curve of FIG. 5 and theerror between the plane and data.

FIG. 7 illustrates another plane fitted to the curve of FIG. 5 and ifthe associated error.

FIG. 8 illustrates yet another plane fitted to the curve of FIG. 5 andthe associated error.

FIG. 9 illustrates three planes fitted to the curve of FIG. 5.

FIG. 10 illustrates the error associated with two of the planes of FIG.9.

FIG. 11 illustrates the error associated with the three planes of FIG.9.

FIG. 12 illustrates a curve representing data and two planes one whichprovides a best fit and the other which deviates from the data by apredetermined tolerance in one sense.

FIG. 13 is a diagram used to illustrate the fitting of a continuousfunction to the piecewise-linear approximation.

DETAILED DESCRIPTION OF THE INVENTION

A computer implemented method is described for the piecemeal orpiecewise fitting of planes to data. In the following description,specific details, such as are shown in some of the drawings, are used toprovide an understanding of the present invention. It will be apparentthat the present invention may be practiced without these details. Inother instances well-known methods and algorithms have not been setforth in detail in order not to unnecessarily obscure the presentinvention.

The disclosed method is useful for providing a mathematical expressionfor systems having virtually any number of variables (e.g., in excess of100,000) using a commonly available personal computer or workstation.

As can be appreciated, it is difficult to illustrate the fitting ofplanes to a system having multiple variables. In many of the figures inthis application, data is represented by a simple two-dimensionalfunction such as the curve 40 of FIG. 5 and planes are represented bystraight lines such as lines 41 and 42 of FIGS. 5 and 6.

The algorithm for function fitting of the present invention provides anapproximation of the form

f(x)≈max_(j)(a _(j) ^(T) x+b _(j))  (1)

where it is assumed that the function to be fitted is scalar-valued andwhere xεR^(n). The input to the algorithm is a set of pairs (x_(i), f(x_(i))) and a pair(a_(j),b_(j)) is a plane.

Initially, it is assumed that data representing a system is stored, forinstance, in a computer memory. This is shown as step 10 of FIG. 1. Aplane is selected which does not cover any of the data. More precisely,a point y_(i)=f(x_(i)) is covered if

|{tilde over (f)}(x _(i))−y _(i)|≦tol.  (2)

Referring briefly to FIG. 5 assumed the data is represented by the curve40, the plane 41 does not cover any of the data. This selection of aplane is step 20 of FIG. 2.

Now planes are fitted to the data, is for example to curve 40, forregions of data. This is done in one embodiment by determining themaximum error between the plane 41 and the data. The maximum error isillustrated by point 51 of FIG. 5 (step 21 of FIG. 3). A region isdefined to include the data associated with point 51 and then a plane isfitted to the data for this region. The manner in which the plane isfitted to the data is discussed below. The size of the region, that isthe amount of data in the region, is selected based on criteria such as,the maximum number of regions necessary to cover all the data, thenature of the data, for instance, how quickly the data change relativeto the tolerance, and the degree of accuracy sought by the algorithm.The size of the regions can vary for each iteration and is automaticallyselected.

After the first plane is fitted the next largest error between the plane41 and the data not falling within the first region is found. This isillustrated as point 52 in FIG. 5. A region is associated with that dataand a plane is fitted to that data. The next largest error is found forthe remaining unfitted data and a plane is again fitted to a data in aregion associated with that error. This is illustrated by region 53 forthe data in FIG. 5. This process continues until planes are fitted toall the data. In FIG. 3 the steps for this are shown as step 22 “selectlargest error and define a region encompassing corresponding data,” andstep 23 “select next largest error and define a region encompassingcorresponding data outside of existing regions.” Step 23 is repeated asshown by path 24 until all the data is encompassed in regions and aplane is fitted to the data in each region.

There are numerous well-known techniques for fitting a plane to datawhich may be used for defining a plane for the data in each of theregions. A well-known technique for a regression fit is used is oneembodiment. In one embodiment the weighted sums of the absolute errorsis used to fit the data. (This is somewhat a deviating from much of theprior art which uses a weighted sum of the squares of the deviation.)Thus the fit minimizes locally

Σweight|f(x_(i))−g(x _(i))|  (3)

Other type of error (e.g. maximum error) can be used without loss ofefficiency provided that the resulted minimization problem remainsconvex.

There is however an important limitation in this fitting in that in noevent can the locally fitted plane outside the region for which theplane is fitted exceed a preset tolerance in one direction or sense asrepresented by

g(x _(i))≦f(x _(i))+tol.  (4)

This is shown in FIG. 1 as step 13 “fit plane to data in each regionwhere plane is not beyond data in one sense by more than tolerance.”

The limitation of equation 4 is shown graphically in FIG. 12. Assumedthat curve 60 represents the data. As can be seen from the sin x—likeshape of this curve, this curve cannot be fitted to a reasonableprecision with a convex piecewise-linear fitting. The data mayrepresent, for example, some data anomaly in an otherwise convex system.Assume further that data is to be fitted to region 63. The best fit isrepresented by plane 62. However, plane 62 is above the data by morethan the tolerance outside of region 63, hence it does not satisfy thelimitation of equation 4. Plane 61 on the other hand, while it is notthe best fit, it is the best fit that satisfying equation 4.Consequently plane 61 is used in the present invention to preserve thecondition of equation 4. As the algorithm progresses, as will bediscussed, each time data not covered by a plane as defined by equation2 is identified, region 63 may again be selected and the fitting processwill again result in the plane 61.

The next step in the algorithm, step 14 of FIG. 1, is the selection ofplanes for inclusion in a set of planes. The set of planes ultimatelyprovides the final piecewise fitting of the data. The set is formed byadding one plane at a time and by testing the error resulting from theaddition of that plane to the set. The plane that results in thegreatest reduction in error is selected for inclusion in the set. Thisis repeated for each iteration of the algorithm. (This error used forthe selections of a plane is not the error discussed in connection withFIG. 5, as will be explained).

Initially, the only plane in the set is plane 41 which does not coverany data as shown in FIG. 5. This plane is deleted from the set afterthe first iteration as will be described.

Assume that the data is initially fitted by three planes represented inFIGS. 6 through 8 as planes 42, 43 and 44. (This is a simplificationfrom the smaller regions shown in FIG. 5 to better illustrate themethod). First assume that plane 42 is included in the set. The errorbetween plane 42 and the data is determined. This error is shown by theshaded area in FIG. 6. The maximum value of the planes 41 and 42 aretaken and compared to the data to computed the error.

Similarly, as shown in FIG. 7 the maximum values of the planes 41 and 44are computed and compared to the data and the error found, as shown bythe shaded area in FIG. 7. Finally, as shown in FIG. 8 the error betweenthe maximum values of the planes 41 and 43 and the data is computed. Inthis case since plane 43 is shown to be substantially parallel to plane41, the value associated with plane 43 determine the error since it hasthe maximum values for all regions.

The plane that provides the least error is now included in the set.Assume for sake of discussion that plane 42 is the plane that isincluded in the set.

As shown by line 17 of FIG. 1, the next step in the algorithm is toagain fit planes to the data. Any data not covered by a plane as definedby equation 2 is refitted. In one embodiment what occurs is once againthe largest error is determined as shown by step 22 of FIG. 3, a planeis fitted to the data is the region associated with that error. Then thenext largest error, as shown by step 23, is determined and a plane isfitted to the region associated with that error. This continues untilall the data not covered is fitted. It should be noted that if forinstance plane 42 is selected and refitting occurs that planes 43 and 44will not necessarily reoccur, and in fact it is unlikely that they willreoccur. The reason for this is that plane 42 was selected for aparticular region. Plane 42 may also fit other data outside of theregion for which it was initially fitted. Consequently after plane 42 isselected for the set and refitting occurs a new set of planes willemerge. This is shown in FIG. 1 as step 13.

Now each plane of the newly fitted planes is considered with the planesin the set of planes and the error determined. The one plane of thenewly fitted planes that most reduces the error is included in the set.Then another iteration occurs as shown by line 17, where the regions areagain determined, and another group of planes is fitted. And once againthe plane of this group that most reduces the error is added to the set.This continues until there is no reduction in error by inclusion of newplanes. At this point the algorithm is completed as shown by step 16.

FIGS. 9, 10 and 11 are used to illustrate the reduction of errorresulting from the addition of a plane to the set. Assume that the setof planes only includes plane 42 and 44. Assume further that once steps12 and 13 of FIG. 1 occur, plane 43 is fitted to a region. The errorassociated with planes 42 and 44 is shown in FIG. 10 by the three areas45, 46 and 47. When plane 43 is included, the error associated with theregion 46 of FIG. 10 is reduced to areas 48 and 49 which provide lesserror than area 46 of FIG. 10. Thus for this discussion, plane 43 wouldbe selected for inclusion in the set. If on the other hand, theinclusion of plane 43 did not reduce the area 46, the fitting would havebeen completed as shown by step 16 of FIG. 1.

FIG. 2 illustrates the steps discussed above for determining what is tobe included in a set. There is a first pass where a first plane (plane41) of FIG. 5 is selected which is apart from the data as discussed.Then as discussed in conjunction with FIG. 6, 7 and 8 the error iscomputed as shown by step 26. Now as shown by step 27 the plane thatmost reduce the error is selected. Steps 26 and 27 are repeated for eachiteration each time there are newly fitted planes. The plane 41 isdropped from the set since after the first few iterations, it no longerplays any part in computing the error.

It should be noted from the above that without the condition of equation4, it is very difficult to devise an algorithm that ensures thatequation 1 will give a reasonable fit whenever such a fit exists. Forinstance, if plane 62 of FIG. 12 were selected, it would provide themaximum value over many regions for which it was not fitted and forwhich it is a very bad fit. Also, if one of the planes such as plane 62of FIG. 12 was globally above the data outside of its associated region,it would be indistinguishable from other planes when the error iscomputed. This would make the selection of the plane that most reducesthe error difficult. Again note that the error associated withdetermining which plane should be included in a set should not beconfused with the error used to define regions to which data is to befitted as discussed in conjunction with FIG. 3.

The above method can readily be implemented with a computer program, forinstance, programmed using C++. In one embodiment this was done withapproximately 2000 lines of code. This excludes the fitting of a planeto data step which is done with a separate program for-this embodiment.There are numerous known programs for doing this fitting step. Theprogram can be stored or shipped in any of a plurality of memory devicessuch as a floppy disk, CD ROM, hard disk, other non-volatile memory, orthe like. This entire program provides good results when running on a PChaving 512M of RAM and a 400 Mz Pentium® Processor. For example, asystem having four variables, 7,000 data points and resulting in anexpression having four planes is solved in approximately 30 seconds.(This is without the look ahead feature discussed below). Interestinglythis would require several lifetimes to complete manually.

In some cases it may be that the inclusion of a plane in a set of planesmay reduce the error by the greatest amount for a particular iterationbut that its selection is not the best when taking into account whatwill occur in subsequent iterations. A method for looking ahead todetermine whether the inclusion of a particular plane will provide goodresults in subsequent iterations may be used. Let n equal the number ofiterations for which the look ahead occurs. In the previous discussionn=0 in that there was no look ahead. As will be seen as n increases thenumber of computation increases dramatically, however the resultsjustify the additional computations in some cases.

The process for n=2 is shown in FIG. 4. Assume that there is a set ofplanes that has been selected and that two planes are been fitted to theremaining data (planes A and B) as shown by step 25. If n were equal to0 the one of the planes A or B that most reduce the error would beselected. With looking ahead both A and B are independently tried in theset as is shown by step 26 and step 27. First plane A is added to theset as shown by step 26. Without any testing for error, the data notcovered by the set and plane A is refitted. It is assumed that therefitting results in two additional planes, planes A, and A₂. Similarlyas shown by step 27, plane B is assumed to be in the set and refittingoccurs. It is again assumed that two planes result from the refitting,planes B₁ and B₂.

Since n=2 and we are looking ahead again after steps 26 and 27. Now newsets are formed, one including A, and another including A₂ as shown bysteps 28 and 29. One of these new sets includes the original set fromstep 25+A+A₁ (see step 28). Another new set includes the original setfrom step 25 and A+A₂ as shown by step 29. Again refitting occurs. Instep 28 it is assumed that after the set includes A and A₁, therefitting results in two additional planes A₁₁ and A₁₂. Similarly asshown in step 29 it is assume that when A and A₂ are added to theoriginal set, a refitting results in planes A₂₁ and A₂₂.

The same process occurs in connection with the set that includes planeB. As shown by step 30, it is assumed the new set includes B+B₁ andafter refitting planes B₁₁ and B₁₂ results. Similarly as shown by step31 the new set is assumed to include B+B₂ and then after refittingplanes B₂₁ and B₂₂ results.

Now the reduction of error is determined for the set with A+A₁+A₁₁ andthe set with A+A₁+A₁₂. The one of the planes A₁₁ or A₁₂ that mostreduced the error is the one selected for the comparison that occurs instep 32. Similarly the error is computed for the original set+A+A₂ whenboth A₂₁ and A₂₂ are separately included. The one of A₂₁ and A₂₂ thatmost reduces the error is the found. The maximum reduction for error isdetermined as shown by step 32 between the error determined in step 28verses step 29.

Similarly the maximum reduction of error is determined by step 33 forthe error found at steps 30 and 31. Then in step 34 the error reductionresulting from step 32 is compared to the error reduction from step 33.One of the two provides the maximum reduction. From this it isdetermined which one of the sets that included the “A” or “B” planesresulted in the most reduction in error. Then either plane A or plane Bis selected for inclusion in the original set from step 25 as shown bystep 35.

Note all the additional computations required to select between planes Aor B as shown in FIG. 4, however in some instances this additionalcomputation is warranted because ultimately with look ahead, planes areselected with an eye towards what will occur in subsequent iteration.

For FIG. 4, if the look ahead is with n=1, at steps 26 and 27 thecomparison between the reduction in error occurring at steps 26 and 27would be compared and one of the planes A or B would be selected.

The above described method can be used to fit a function to amax-monomial approximation by using the log of the data. The result isthe function shown below by equation 5

{tilde over (f)}(x)=e ^(max) ^(_(j)) ^({a) ^(T) ^(_(j))^(log x+b}).  (5)

In some cases it is desirable to differentiate or integrate anexpression. When integrating, for instance, a piecewise-linearexpression the result is not a piecewise-linear expression. A functionthat can be differentiated or integrated is useful, for example torepresent electrical circuits, and more specifically a posynomialexpression. Taking advantage of equation 5 a new parameter ε can beintroduce to provide the function equation 6 $\begin{matrix}{{{\overset{\sim}{f}\quad (x)} = \left( {\sum\limits_{j}^{\quad}\quad ^{\frac{{\alpha_{j}^{T}\quad \log \quad x} + b_{j}}{ɛ}}} \right)^{ɛ}},} & (6)\end{matrix}$

if ε=1 equation 6 is a posynomial. As ε approaches 0 the function is amax-monomial approximation almost everywhere.

The expression of equation 6 can be obtain from equation 5 by startingwith a small positive ε and gradually incrementing it until ε=1. At step“a” and “b” terms are adjusted to locally minimize the 2-norm of theapproximation error. Since the function is differentiable, so is theapproximation error, thus the local minimization can be performed usingthe gradient and the Hessian of the error.

An attempt to show this process graphically is presented in FIG. 13where the piecewise algorithm resulted in the planes 70, 71 and 72. Forε=0.1 the curve 73 is shown. Adjustments are made at this point tominimize the error so that curve 73 more closely approximates thepiecewise approximation. As ε is incremented again in each of the stepsthe terms are adjusted to minimize the error. Curve 74 shows thefunction with ε=1, before the terms are again adjusted to obtain aposynomial. The process of incrementing E and adjusting at eachincrement is necessary to obtain a good fit in most cases.

Thus a method has been disclosed for providing a piecewise approximationof data for a system having a plurality of variables.

I claim:
 1. A method for providing a mathematical representation of asystem having a plurality of variables, comprising: a) collecting datarepresentative of the system and storing said data into memory; b)identifying a plane which does not cover any of the data; c) fitting aplane to a region of the data for each of a plurality of regions thatthe data is divided into, where, each next fitted plane is fitted to aregion that i) is not yet fitted to by a plane and ii) possesses themaximum error in the data, amongst the regions not yet fitted to, asmeasured against the plane that does not cover any of the data; andwhere the difference between each fitted plane and the data outside itsregion does not exceed a predetermined tolerance in one sense; d)selecting one of the fitted planes as a first plane for inclusion into aset of planes, the selected one plane being the plane having the leasterror; e) fitting a plane to a region of the data for each of aplurality of regions that the data is divided into, where, each nextfitted plane is fitted to a region that: i) is not yet fitted to by aplane that has been selected for inclusion into the set of planes and,and where the difference between each fitted plane and the data outsideits region does not exceed the predetermined tolerance in one sense; f)selecting one of the fitted planes from e) as a next plane for inclusioninto the set of planes, the selected next plane being the plane thatmost reduces error by way of its introduction and repeating e) unlesssaid next fitted plane could not be identified because no reduction inerror would result by way of its introduction.
 2. The method defined byclaim 1 wherein said fitting further comprises using weighted sums ofabsolute errors.
 3. The method defined by claim 1 wherein said fittingfurther comprises regression fitting.
 4. The method defined by claim 1wherein each region of e) where each next fitted plane is fitted to isfurther characterized in that it: ii) possesses the maximum error in thedata, amongst the regions not yet fitted to, as measured against a planethat is substantially parallel to the plane that does not cover any ofthe data.
 5. The method defined by claim 1 wherein e) further comprisesfor each next fitted plane: looking ahead to a next iteration, saidlooking ahead further comprising fitting to regions of the data notcovered by the set of planes and the next fitted plane.
 6. A computerprogram product comprising: a computer usable medium having computerreadable code which when executed causes a method to be performed, themethod comprising: a) collecting data representative of the system andstoring the data into memory; b) identifying a plane which does notcover any of the data: c) fitting a plane to a region of the data foreach of a plurality of regions that the data is divided into, where,each next fitted plane is fitted to a region that: i) is not yet fittedto by a plane and ii) possesses the maximum error in the data, amongstthe regions not yet fitted to, as measured against the plane that doesnot cover any of the data: and where the difference between each fittedplane and the data outside its region does not exceed a predeterminedtolerance in one sense; d) selecting one of the fitted planes as a firstplane for inclusion into a set of planes, the selected one plane beingthe plane having the least error; e) fitting a plane to a region of thedata for each of a plurality of regions that the data is divided into,where, each next fitted plane is fitted to a region that: i) is not yetfitted to by a plane that has been selected for inclusion into the setof planes: and where the difference between each fitted plane and thedata outside its region does not exceed the predetermined tolerance inone sense; f) selecting one of the fitted planes from e) as a next planefor inclusion into the set of planes, the selected next plane being theplane that most reduces error by way of its introduction and repeatinge) unless said next fitted plane could not be identified because noreduction in error would result by way of its introduction.
 7. Thecomputer program product defined by claim 1 wherein said fitting furthercomprises using weighted sums of absolute errors.
 8. The computerprogram product defined by claim 1 wherein said fitting furthercomprises regression fitting.
 9. The computer program product defined byclaim 1 wherein each region of e) where each next fitted plane is fittedto is further characterized in that it: ii) possesses the maximum errorin the data, amongst the regions not yet fitted to, as measured againsta plane that is substantially parallel to the plane that does not coverany of the data.
 10. The method defined by claim 1 wherein e) furthercomprises for each next fitted plane: looking ahead to a next iteration,said looking ahead further comprising fitting to regions of the data notcovered by the set of planes and the next fitted plane.